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DEFINITION, NOTION OF STOCHASTIC PROCESS 



1.1 






[ 
|S$|j5d6 (random process Or Stochastic Process) <r 






t sStfSb 836A?6 fS&cabs^, (0, t" 6* 



55 




X 5* ?Sra^, S'o^^^o^ 8 t 9 &^Sg X t o ?3sSDj-^o ^&&oa. t 



T 



lj, |X fc . t 








, t E T, <) E S\ 

co 




X '. 




x (to)-x t s t 

X '6>V c3i)^^_ LSSS^PoS (j$3ocSo&a (sampling function) 
'realization) ^ odp^D. cutf 5*%$ (die) a flTQatfrfoff n S5 
SOo*!^ X S ^r>01j f X feS)9T^ 6otfS Sco^o ?3Sb^ S = | 1, 2, 3. 4, 
6, 6 '.. 3, 1, 5, 4, 4, 2, 6, 1, 6, 3, ... o& ^^ a.S' 
L ^ L CJJO, {X , n = 1, 2, 3, ../, ^^^ aa^ff [typical) 

X - 

g (o, t 



t 







5*3 t 



^ (mgw) 
t * X 






t 



, X 



X 











t &&& .&o"&, t 5* X (w) 




fState spaced : X 
v / 



(states^ 

\ / 






infinite^ 



(i) 
Hi) 



fstate space") 

\ , J 

(Countably 



s*>d3 ^60^23^3 






(ii) 



S;sSa. (0, t" 







(0, t" 

\ * . 



ftd 



| 



x x ^ O 



J 



t ^ OJ 

~ JO, ^ 1 2, . ..j. 

tj 



X 






(Index set) : 




, t 8 



X o 



(ij 



(a) 



T ^safcd^ ^^^ (countable setj 
Jt t = t nf t., t , . . . , t ; n 

^Ul^Sn 

o |t t = 0, 1, 2 .. 
y |tt = f + 1, + 2, 
fc , t T| 

-J 



T 
T 



|X t , X t , ... , X t 

(. o 1 ' n 



r X gl ... 



} 



IV\ 

' * 



.' 

i 

i 



t a ^ t ^ b; a, b epo sp rS 

--' * - 



co 



L s 

-jtO ^ t-< ooj, gey jt 

/X~, t TJ |jSL8d6-^otf-;6sSbasS'SS $dboa 

w ' 

JX f t TJ *& t a s$oSbd esfc, T 

(I _J . , -I 

; T rtra>c crSia JS&8 escoi^, t a aa^sSj, sSo-Sca o; T 

- " * ' "*^ 

0*3^ jsaia'oo^; 't a <fca "efo-acasSa. t 

T tf^aa 

T - 

-^ t 






t, . ?$jo^>, t s_- 



. x, 



t 





{x t , t s> oj 

*^ (cross section) 3 
, t, es&ro'S, t 






X 8>r SeoS X 



T, 5 X t 






(Probability distribution): |5 



t ^d X 






Stfoaotf&fcy s, 



t 

T d t 

t EX. c^) s O^ L^L d *a- x 

f6od3b_^ SE^ssSS^ (joint distribution) n^o* 

, t E 




3 



X t X t ..oX t ^ n (< co). 

L lf Ig, ^ V 



*tT, 

(liiiear iiidex set) 

<n < oo 



S 




(a) 






(i) 







t 






s X . 



t 



, X . 




(c) 



(d) 



(i) 



H t 






0) 




X . 





X . 







n 



X. 

t 



, X . 

t 







: (i) 






JfoS fl 




(ii) 




t 



t> tT } 






T = "0, oo) 







Q t 



1.2,1 <Bj>efoeF Ibcft/Ctfoaj/fo U$/j!d&Q} "(Process with indipendent- 
increments) : T=~0,co) &o 0<t 1 <t g <...<t 

X Q =0 <sos>, X t -X t ,...,X t X 

, t E TJ 

n >0 So^sS Sg_?^ sS6 
. sa > T= JO, 1, 2, ...| 
t , t E TJ 




Z Q = X , Z. = X X ; i = l, 2, 3, . 





sequence 




X. 



t = 0, 1, 2, ... } & 

Z.o ^Spod (marginal) 



fc , t = 0, 1, ...j 



C9 



U$i_Sc&su (Markov Processes) : T &DO t <t <t 

' ' lo 

L?58 *! V l s ^' ^^^ ^o^^g^- (conditional pro 
bability) P (X t A X t = Xj, X t =x 2 ) [Sfe x l 

3 1 3 



P ^X t A X t Xg) & r6&*sS^S, wdpofi (j$L_ifc { x 

S & * 



t T j &> sSi^e^^S (jS[d6 (Markov process) 
{X t , t T} sip^S j5 L c6 ^COD^, [SSL 

t t 



3 



n 
, T 3oS, t. < t < .... < t < t u 

id n 

t jf ---,t n , t o^b o$osDoSoa ? &d e^^d'ra S 5^(0 [_^S^ x x , ,. t , x 

T } &> 

(X ^a X t =x.,..., X t =x 

1 n n 

= P(X ^aX n = X n ) ... (1.2.1) 








A X = 





L :S L 3csfi o L $r& SowSgo* iS&dfi&o (Transition 
probability function) w 5>e^fcj. ^ (1.2.1) 




P(X^aX t = Xl ,...,X t = x) = P (t, V t, a) 

1 n 



(X, t E T= "t ,00)} tig<$o[g "^db^^oexj^o dScp^S)^ |oS[_ 



cJi) 



b, P(X t =b) = l ^o^ {X t 






T Z 

ex* 

n 



x =x t + 2 (X t 

n m 



n 



= b + Z r? 



m=l 



=X t -X t 

m m-1 



(X t ^xX x ..., X t = x n- 

x n 1 n-1 



PX 



= P(X t 

n n-1 






P(X t -z lf X t =x 1 +x 2 ) = P(X t = Xl , X t -X t 

1 & , A 



t 9 l 

. 3-S ^ 3^go35 $S^&5 
P(X t = 0) = 1 S* (X t , t E T = "t , 

<tj <t a < oo esAj&to T ff'a ^ 'So^ ^O'Sba So3Sk V t g So 



8 





CO 



2 






ingale) 






. 5&:Soa ^ L dt3i^) ^SoT^eo (marti 

[martingale equality) 

n , n = 1 , 2, . . . I |_s 



E (Y n ) = 0, n > 1 ao^ X n = E Y. rs 



{X , n = 1, 2, ... 

\ ^ 



( X j.1 X =x ..... x, = x 

\ n+l n n' 1 I 



==E ( X n 



""~ X 

n 






X n+i 



n + , n~l, 2, ... 



1.2.8 c2>o??a;j (Martingales): 

, T a* t- < t < ... < t < t Sorf^s^S (ssa K$a>a 

1 o n n+1 cJ L. 

tj, ... , t n+1 | rr&>, S 
X t , t E T} go 



E (X t X t = x , ..., X t =x ) = x ... '1.2.2) 

V n + I l l l n n ' n ' 



x 'n 1 









'fair game) 



E ( X - X == X , j 2v t == x ) x . 

V n+1 n n * I u n 



t c$&cfca 82^6 7^coS3)?SD X S* ?3jat), f 1.2.2^ iSjs^^o t 

t \ / 



x 

n n + i n 



, ... t 



X t , t ^ LSS L dSi ^O L ^ -^&^tfoa> KOfl, P (X Q = 0) =r 1, 
t > 0, E X < oo, E (X \ = foo'og'&^P ,sSS /X , t ^ 0\ 

t \t/Q (DL^t 



X = X t , Y g = X t P X t , .... eo 

1 21 




n n- 



( x t - 



n 

n + 1 

Y. 



' * .* == X j ) X.A ^X > 

n 1 n n -J 






= X . 

n 



Doob 
M Stochastic Process" 5 



1.2.4 &>& iSS(d&&} (Stationary Processes) .- {t t g , ..., t " 
T 5*a &8* E.S' n 



h > L^8 rSSbd { t 1? ... f t } , (X , t T} 0*3 (X t 

-f-j^}* X^ f X^ ,,.. fl X" I ^ "3o<& ^00*^)0 ?6So^be a."! 

n 1 2 n 

g_^, {X , t E 



i 





"(1.2.3) 







(1.2.3) 

es^tf'SrfS', t ^&<en X ofe^. a, 

W- t ' W* 

(1.2.3) L$5>tfo X 







P (X fc ^ x) == F (x; t) = F (x; 0) =-F 




n=l, h= t 

h>0, t T S 




C(t, t+h) = Cov (X , X t+h ) = E (X X t+ld -:E (X) E (X +h ) 

= C(0, h) 



; <9oe, ^ij^) C (t, t+h), h 2^ *av%3& AofapoS*, - 

i 

(wide s,ense\ ^sStf 

\ / * 



1.8. 







"38^1) (jo[d& (Bernoulli Process) : 



esi3 "60^0 ^Os'e^rt'o (Sjg^o^ ^J6o*s5^ [Jodie 1 ';) o 

^d ^^ ^^ c ^'. '^So^ J o fSo5PSSgtfo& .SS^iT P, q 
p ^ 0, q ^ 0; p+q=l w^ooo./n' LSSd&o^ofiT* 

-l' ? 2,...) 




n 

fc" 



P /S = V\ n* /i n ~k t n i > 

In K j v P q . . k~ 0, 1, 2, ..., n 



'II 



JS , n= 1, 2, ...j &3 dScp^^ar- tfotr&o &r>*&y |0, 1, 2, ,.. 








Bernulli process) 



(k 1) 3 ?5a&: eftfsptf k 

, k 1, k 



P(W k = r) = q'r-lp, r =l, 2. 



W, to fSorf iS"^^) a.'S ^lbfed& ^eaSSsSoa /'Geometric 



distribution] 

. 






n 

n & o> n 




P s = i S. = i., ...... , S =i 

n + l 1 ,1' n n 

= P 'S . = i S = i } 

\, n+l n n/ 

(1-2.1) 



S = JO, 1, 2, ...j tfo sirS^S |jS^8d& 

^rZ3 Sj*e5'ra^b, S^y 6 S50DDS n sSf&Sefir 8 <<c <5* ^?ODOD D 

* 

, n= 1, 2, ... 




1,3.2 &av&*($~ [Zjjdfc (Poisson Process) : sjfi 



fcS 




(a) 



, X 



(i) ;P ^t f t 




(ii) P (t, t + At" & Sc'S)cS 6toSo ^ospg >l = O 

O^Atl 

At ~> ^o^, v y - o as&&otfs o (At) 

()' (") ^el^S- 





P ' (t, t+ At" S* fSotf>o*! &&&D SSotfg = OJ 




(0, t 

t , t ^ OJ dtfptf^ 
S = |0, 1, 2, ... j. 5 (a v , (b) fc5^ere?S&<Sao^ L?36 X fc (t ^ 0) 



P(X-k) =c-" _ t k-0, 1,2, 

\ t / k ! 

iSrSsSt^. eg {X t5 t ^ OJ &) SPOX- 

X, (0, t' fi^ &g c&p^dS" s^oo^ ?6o^So^ ^)fio?5o f6Kdx>"3to (mean 
rate) 



(a), (V S>otf^oo (SseJo (jSS t, s > 0, 1 1 s, (0, t", (s, s + t" 



X X i X X co <DU* a* 







(X fr -X =k)=e- t _, fc-=0, 1,2, 

V s+t s / 



k ! 
P (X - X n k) = P ( X - X = k\ 

\ t J \ s+t s / 



(X - X } = Xt = Var. (X - X } 

Vs + t sj \ s + t s f 



..j 

i S 



5 t 

1 




(t > s) 



= As . X (t - ) + (Xs + X 2 s2) - Xs . Xt 
= Xs = X . min s, 



t 3$(fc&) [jSS'tj, t g , .. . t , t 



< L < ... < i ^c^)9|^d i , ..,, i , i 

i n i)* L n 



es 



PfX =iX t =L, ...,X t =r=P(X =TX t =i 

V * n * n n ' V f n n 



(1-2.1) 
0, 1, 2, ...J 






e) tJ 



1.8,8 (9joip^ " feijd&lu (Two-valued Processes) : 
.38^1) cScptf^JD^ tfoo*&a> f5osrSSgo 




a, 







g (Two- valued) |j5(Jd&podj*3b. (jSSjSoO U 1 

{X 9t^'OJ t & "1, 1" (One - minus 

One) L55[3c& <sotp>Co. |X fc , t ^ 0\ ^a "1. - ' 

* 



b + a b a 

Y t= 



Y, t^O |SS L ciS) a, b 

t < ^ 



. t) 



. t = ^COD^, N =0 ; t > esan^, N fc a ( 



0, t 






j N fc , t ^ . 3 SS q5*gdi)o^ atfjjQoarS rte?S |jS|8dfi ("counting process) 

^ . m 

, t 




X =X 

t -o 



N t ... (1.3.2) 




t ^ 

' ' 

(1 3-2) fir* StfgaoOS -'X^ t S O. & dfp^^tf "BOiji-5 




14' 




(random telegraph signal) oofcr>&. o"S, '1.3.2) 

_ - . / . . . k 

, t ^ 



(i) 

i 

(ii) 



p ( x o= i )=i= p ( x o=- 1 ; 



|X fc 







x t = 



(zero-one) |j$ 




^ repair 



"off* fod*S ^otooa. ft9Sjv(& t 



X t 











^^ 

Too ^tfSb ^S^dabSiS. "off 1 




o 




U; 1 _ > 
V 



, U, V 




Lt P "X =0" = 



' ' . .'ECU) 

" l ' 




Lt P ~x = r = 





3.9.3 



1.&-.4 '7P8d&<?" 



(Gaussian or normal 



process : sj<3 






X r= 



t r 

* t' - 

_.r> . 



, 2, ..., n 

" .- 

... 

normal 








o 



0,, .. 



iS5icdo^i 

1_ L_ 

I 

o (characteristic function 
n 



,...9X4. o 



x 

ttf r * ' '*' t' "" 



... + e n x t 



n 



n 



n 



n 



exp i 



i ~ V 1 
1 3.3) ^ 



E (X t ) 



/X t 

V 



6 e k cov.(X t , X t ) ^ (1.3.3) 

" 



> t ^ C. 




E i X 



L. 



t X 



i 

L 



Ga'iissi&'n series-" 







'fio)S "850 [TV 5 



(Weiner Process) 















iS$,d& i X t t ^ 1 Basis' SiSS" 3o isr^di^ 'Wcincr or Brownia^ 
L L I t* f L 



, t ^ 





'ii) [38: t > &, X 





(iiij X = 0, E (X t ' =0. V t > n-& AorfsSB. ^ 
t ^ s ^ 



X X 

[ 5 

. X 5 X cx> ( 







c ( S| t) = covfx, x) E 'x x; 

\ / \ S t/ i, 8 t 

- E |X 'X - X + X * '. 

; ( k t S / , 

' - E X (X - X ' + E (X 2 ) 't > s) 

5\t , \&/ , / 

x ... a a ' " 












(Brownian motion) 



eDcp^or^ tfr&?6^dJD 1827 






(Robert Brown) &3 &oti **^~3>^ S'fSaj.sS^osSe) es 



(0, t" 6* tf>o>tf iiptfo X t 
X rr* 5^S)otD, fs. t" &3 

o V 

X X . fs, t" J^^) eiSh XE^C^^ ^ou)?5 
t s \ _ ^ 





_ X X 

/fcrl ^S S)^orr (s, t" fiT 1 tfOoCtf i&^2 f X X fl 

o * s + h t 




17 







= t s 



t + h- s - h 

,.aol 



^Cfc/fctf ceo 



h > So X ^ X v X f 










Cj 



S^sff 

(j5ic (Brownian motion 

*"* .V, 



Process) 



2 






(MARKOV CHAINS) 



t 



OO 



(Physical system V 

\ J J 



(State) ST 8 &0^ SSScPiFtfo ^CFtforF 5?3 S^e&D t. ( > t 

\ / 1 \ 



S$gsSrS 



( 



t 






t 






o 



&D t < t 



es <s> 

t 




2.1 




125(3 dfib 

t , t E if, T= Jo f i, 2, ...J go- |t 



t. n 

U I L. __* L_ ^ 12 



I25& sr ^SS Dexi^co x x ..., x 

*- B i 2s n 



T ^ n + 1 0*338 S)euo t p t gj .... t 



, 

+1 



P (X t ^ x X* = x , . ., X t 

V n+1 l [ 






t 



n - > n - 



x 3fi"&, t &ctfra ~X t 

- 




S X &* Aotfcrai 



. t m (m < n) g L sS L dS d^ |d (xj 
(2. 1.1) ^5*^5 ^d6s&o (Markov Condition) 



(Conditional distribution function) 



F(t 1)X; t a .y)=P(X t 

(2.1.1) 



... 



(2.1.2) 



t < t & oSDoSoa t o 



t , t 

n 








= t a " t 1 ( l i < t a) 




F(t p x; t a y) = F(t, Xl y) ...2.1.3 

(homogeneous) 



P(X t = = =x ...2.1.4 



& * 



2.2 ^8*5-1 jS5|SdSje - 

ijS[_dac>G^ ^0^ ?63DS. fcS esSStfwe O^ES-O 




co 





20 






-figs' 2,2 



^l. c . 

( -2-1 1 2 1 S:{-co<x<} 

. < , ~ 1 v> J j A., ... > 



' O 

69 



T: {0, 1, 2, ...} ^5*^5 *^o$oSS & 5*^.5 








X A , X,. ...., X , X , 

! 0* 1* n n + 1 

OJ<>, n 



X 5>&>^o~&> X GO cSScpdJ^O^ 00*360). JX , n= 0, 1, 2, .... 1 



n esc<?(b(iDD, n 1f ...., n es m 2 o8peoPr> f x,, ..., x 

m o 1 m ^ E> 1m 



X n ; ! = x X n =: x_, X n x > . . ,, X n 

n m m D D l X Q m-l 



= x 



= P(X n =x X_ = x ) ...2.2.1 

\ u _ m, ** __i m-iy 



P ( X m = X m X = X ' X l = V ' ' 

...2-2.2 



. . 

tn m m 1 m-1 

= 0, 1, 2, .... j & sSr5*^_5 ^ojpes&S) ^oipCb (2.2.2) 




, n = 0, 1,2, , . , jj$L3dto (2.2.2) 



X 

U v. L. t V CX. 



s 

V oLa q) " 3 

X" * f\ *t f*\ ^ J\ W -JQ ^^ jg^ ^jF 

= ij I : = ...., 2 9 1, 0. 1, 2* ,,,. ^OOOQ ioiscia> t 

t ' U v_ 

i ss ass* 



* 



= p 



s 



(2.2,2) 6*S X-SOD^G^ i ( = 0, 1, 2, ... ) 

/ 



- 



- J X = V X ! - V ' 



...2.2.3 
V J i " ^-2' ] ' 








2.3 c5oi 

(Transition Probability functions] 

^oe!S n - a eg (step) e^ j 




p. ln) S* 



n-sS ^(^ sS^si (absolute) 

. n 








^marginal 
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distribution matrix) && C<LP&D. |& $8 i S So P (X = i] > 

' ^ $ V n / 



- = Pi, 



lb, p.. (m, m + n 



. m~s5 sto S 

Q O (J fiy (J) 

m + n -sS afi gStf j sS &09 ?So L ^^o^ (transit] 

n 






^o<2.. (2.8) ^ |j3^d&sio S^^PCO^ ?6oi L ^6)e3) (transition) 
(initial) fed i, wotfg (final) fcfi j ^ 



m 

' m 



n m = n) 

p.. (m, m +n) = p c+n-m) = p ..cto ...2.3.^ 




i, j 8 S e&, S Spfi^^S 1 ^ospcpOi (2.3.2) ^ 

?6^&c& ^5*^5 y^ojposSaD (homogeneous markov chain) 
p.. (n) o& ro^Sd& ^o ^P^ ^o^sSa fSoB-ig^ea (stationary transition 



probabilities) 






n 



P(X =jX =i) = p..' n) =PfX =j X =i 

V m+n J m / r i V n . 



p.. 

1J 3 



(one step transition probability) ofe, p,. (n) 







^ojpoSSx> { X , t ^ 1 Si 

fe t ^ go, t ?6& 
p. (t) = P (X = j 
0) SsidfrfS i |8&o t (> s) 






p. . (s, t) = P (X - j X s = i) 
|X fc1 t ;> J fS^Sdfi fir-e^^S ^owosSxxoS, [_sS& s ^ 5o, 
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(x = j x = n = p..c'> = p (x = j x = r 

V s + t J s / r ij V t J 




iSS" (Transition Probability matrix' : 



P= P .. 5 



Q-8 



/Poo p oi p oa ' POJ 



p io PII P i2 " " 



P= P 20 P 21 P 22 " P 2j 







. p io p ii p i: 

\ : : : 

Sfc*|d P3 

(1-step transition probability -matrix) S 
(elements) p, o<5 

P^ 



(ii) S P 5j = 
JES 



(row) 



n - 

P (n) = . ln) i, j - o, i. 2, .. 



V. ...2.3.5 



'p ( n ) 

F oo 

n ( n ) p ^ n) P, (n) P;- 

PIQ F U F 12 j 

f -s *> ^ fi 

p(n) -n tn) D cm p^ (n) ... P 9 : 1 J ...^o.o 

"^ J POA Fgi ^22 2] 



P w 



10 

* 






i 

\ 

- a- L Wf (n - step transition probability 
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matrix) WO&P, <& &j*o3*eo p. . (It) 
/ i j 



(i) p..^ n) ^0 Vi.jES -> 

w 1J w ...2.37 

ii) E p.. tn) = i VIES i 

k / 11 *' 

jS 

, p (n) , P (n) 




|jSS ^ycpos^ex) n, m > 0, [Sia i, j E S oSb 



( n+m) _ pW.p Cm) ...2.3.8 

^jj r r 

kES 



(2.3.8) a r>5a>S" - S'^i^S' (chapman - kolmogorov) oSS 



kES 



kES 



c 



P(X j X =k,X n = i) = P(X L =j X =k) 

\ n+m J n / V n-l-m n / 



= n = t>.. w 






Sb ^Sog'd'sao^ iSS'iSoa fip"fc (-2.2.8} 

- l^ - V / - ^ V 

n, r n 2> r > 0, 5S i, j S 



kS 



. 
=0, i ^t j 
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(non-homogeneous) 






i, j S oSb f [Sfe Sfr>g*o3'e r, n,, m (0 < r < n < m). 



p.. r, m) = 2 P ik (r, n) . p kj (n, m) ...2.3,10 

kS 



P, P tn) 
B==/b ^ eo sSd 8 ^ pxr, rxq 

\ kj/ 

oo SSplD^ C = AB== fc..X fi 63^ pxq. 

\ 1 j / 



Q 



Q 



i, j sSxr-o^^) c.. = S a., b , eso"^ c , A &*& i 

11 IK 

k=l 



'sS S'oSb (column) o OE3 fS&^^o- ^ 3^c6s ^g (2.3..9);..4> 

i JS ^o , P tn " r) 



cp 13 

P tn) 



= per) m pcn-r) ..'.2.3-11 



pen ^= pa 



p(8) p(25 m pd) = p8 



p ( n) = p(n-i) m pel) ^ pn 



(2.3.12) ^Srg n-^to &o\g& rfo^sSg^ 6 S^jJBS' 1 - a&> 




P 3)1> (2 3 12) ^3Q'6' 



P (n) , P 



P cn) 






n ^606-^ sSjg*^_S ^ospoo 



p,n, == 

iES 



iS 
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= S PJ ( O> 
IBS 



t p (n) ar*a j sS Scpos'&i p.< n) , sSo &*$Ef p<o> f pw 



= p CO)pn ^_ 2.3-13 




P (n) ^^g (2.3.13) 
, P 
(n 

(initial distribution), P off* ^efsb ?6o5PSStfe p. (n) 
n > 0) & 





S = 






'1 - a a 

P= ,0^a,b^l, 1-a b <1 ...2.3.14 

b 1 ~b 

i 

fSo^^ ^o^gsp r[&g P (n) ^> S'^j^^o 



(2.3.14) 



oo U1 - POO = l - a ' - - a 



2.3.9 




= b + (l-a-b) Poo - 



b(I-a-b) + (l-a)(l -a-b) 



00 



<> 



* * I 

= b + b (1 - a - b) + . . . + b (1 - a - b) n ' 2 



n-2 



(l-a-b) r 



1 a b < 1 



oo 



1 (\ a bY 1 " 1 

= b _ _ \ _ I __ + (1 - a) (1 - a - 

N ^ A 



b (1 a-b) n -i "(!-a)(a + b) - b" 



a + b 



a + b 



a b 



+ 






(1 a -bY". 

^ 1 



P 10 (n ' 



in 

10 



+ p cn> - 1 
~ p ' 



c) = 1 




P (n) 



P (n) 



, u 



b a 

+ - - (1 - a bY 1 

; 



a + b a + b 



^ a + b a + b 



a+b a + b 



.6.3.15 



a b 




(1 - a ~b) n 



n 



: 



t a b < 1 '*s?gofl n 



oo 



0, 



/ 



a + b a + b 



a + b a +b / 



.,.2.3.16 



2.4, 




) 







n (< 







n 




O 







X o 



P ( X = V X l 



CO 



V 



2.1.1 



X n-i = 



- = P 



n- 1 n 



Pi i ...2.4.1 



n 



[2.4.1) a 




=p(x-=i/.p i j . P j j .-.. Pi j 

' .. . V .. :.: ' 01 12 n-1 



... 2.4:2 





n < n < ... < n, 



tf>o*>oc535. cS 

P (X = i Q ) S)^ X QI X l 



x 



i,, ... i r 

k 



P, X 



o 



.^ -v.n k to Jc 



, X > . .., X o 

n i n k 



]]S8 



J J "* o c 



= p -( x n 1 = 



( x n, - 



= 






2.1.1 



) 



( x n - ! * 



- 



- 



= P 



fc 



...2.4.3 



2.4.3 



- ! X 



n 2 = V 




P; : Pi 



1 4. 
...^.*. 



(2.4.4) 
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(2.3.12) 




2.5 








So 












P 

(2.3.13) 







p (0) , P 




(0) 







n = l f 2 f 8, .... 










P X=^ = 



. X co 



= a. f a. 

t i 



0, 







E3 



i, 






._ 



= 



Vj<i. 



n = 1, 2, ... J ?5^ 
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a o a i 



p = 





X - 



n , n = 1, 2, ... - 

S= .' ..., 2, -1, 0, 1, .2, ..." X n e rf&SumS&D P X = T = a., 
i = 0| 1, ...; a. ^ 0, E a. = I- & SS 



e umD 



P= ... a_ 2 a_ t a 




\ * ' ' 

2.5.2 SUfSv&if 'dfrefjtyJf- *f (random walk)* . 
(particle) 





esoto-, 6SS 5*^.5 




(jumping) eu1 S 

X Q &, n * 8 (jump; 






. n- 

n = 0, I, 2, ... 

i S5 ROfl* 




CJ 



Spitzer (1964J 




p, q (p + q = 1 wc&gfcto) esoa^, [jS8 jfr'Jg'oS'o n > 0' 



P (X n = 1) = p, P (X n = - 1) = q 

X "Sj aSM) Se-fflSFSv w$?58o3. 

Jl ^^* fc - 

{ S , n = 0, 1, 2, ... \ & ^o-sScrc^^ dScp^O^ ?5^ (classical random 

' 

walk) 
I oS^t> 

. = p, j == i + 1 

4 

- - q, J =1-1 

= 0, j-i 





unrestricted random walk) : 



_ 
, n = 0, 1, 2 ... 



n 



h > &, 

P X 



j 

== P 



* 

n-)-i 



2 
n = 



P.. n) ... 

ij n + j i 



\ 
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n 



= o 






POO'"' - (P<0 n/2 

n 2 



' 2n 



p 00 (2n) = 



(pq) n 






(Sterling) 






(4pq) n 





c$#3' ^resricted random walk' 








a 



es 



0, a 
0, 1, 2, .. ., a 

o "\ 

S f n = 0, l s 2 ... I 



2.5.1 










S =: / 0, 1, 2 ... v a j 

* 



rvS, a 








0, a o 



(absorbing barriersj 
, 0, a o 






POO = 



0, a JS'b 




t>;6 



(absorbing 



34 








0, a 




y 1 2 3 ... a 2 a 1 a 







1 



3 



1000 



q p 



q 







y -. a 




x, y caej 



a I 0000 



0000 








, A 



n 







000 



000 



000 



X =s x 




q p 



1 



- A, B 






X 






n 



A cSS>_ 

u 



S = 



B 



X n + X, + ... + X 

U 1 n 



A ' 



B 

ruin) (J$[_So& 



a 

(ruin) sis5^o; S = a 
n , n = 0, I, ...1 ' 



S = 



' /^gambler's 






'ii) 'do A &fj/Tto c^^o; (reflecting barriers) tfv 
0, a 




a - 



" 



a 



"" 



"" 



6fl " 



1 

, 0, a 
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s *oo> (reflecting bairrers) 



= l < p aa -l = l 



a 





1 2 3 ... a la 



o ' o i o o ... o o 

1 q p ... 



... 00 ,..2.5.3 



a_L 1 00 ... 



00 ... 1 



2.S.3 cWjiCf^ ^ (queue) 



(service) 

\ 7 





queue e 

' (servers) 




. sago 






n ^ 1 ^oaofioa n 6 ss-a-cySb customer) So t)S5 Mfi^fioSSj^, esoe!> n 26 



U U , U , ... 

1 2 o 



eo 
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oo 



U= fc) ss a k ; fc~ 0, 1,2, 







n 





X 



, X,, X_, ..., X 



n 



, n = 1, 2, .. 








( 





= x -r 



X 

n 



fn + 1 

\ , 



co 






X 



n+1 



X 



'**%-* 



J(X 

\ n 

1, 
= 0, 

* -' X 



U 



Al 

n+1 



. ..2.5,4 




= 



n 






X 1f X , .. M 

i J 



t 

1 



o 



/X ,n= 1.2, ...1 

e> (^ n > 






- i > 








X = 



P/X ,= 

V n+l 




X = 

n 



n 



, 

n+1 



g /X , n = 1, 2, 








3 Z> 



3? 



a O 



p = a n a. a g .. ...2.5.5 





a 



Q j 




(branching) 







<Sr62r>cySio. J)g_^ s.S' e&D ^ ^2)^goo (life time) 

opeso^o^ 'off-springs) 
^, Z 




00 



= P (Z = fc) - a k , k = 0, 1 9 2, . . . ; a k ^ 0, S a fc = 1 





&S ?6oE*^e3;6 S^diSio probability generating 



function)* g (z\ = 2 a k 

k = 



es?Sa> 5peso&o &os><? X , wo^> 0-sstfcf (generation^ 

t> o V / 

C * m~ ^ ,* ^r IP 

cJ)O f^O^D? X QflrOD^OC^oJCO. ^6 X 
^ a o o 




X 

ea v _ 

CD 

P/V =t\:^o ">n Vfl =1 Mrra Z <T 1 

I ^V ^^ J I ^~^ d, <^- U* ^J a. ^^ X Cp^J-Ui *J ^^ A. 

\ / k JC 



CD 

S ^^ -L 

cjoso wou*5 z abas ; Sj^?^ a)oD:6"B f\5 2j a, z 





X 






n-, X f ^ Seo?i^cr>^ X A , X., ..,, X 

n + l * n-J- 

fi JX , n = 0, 1. 2, ,..) 

ea ( n J 




n 3o8*> ^SPESO^C ?6o^p X = 
o r (r - 1, 2, .... i) 




1 

= S z . sss ^ o 

r = 



i 



P/x =j x =i)pf s z =n 

V n + l J n / V r J / 



1 



1 

i 

X^V ^ * V^4 

, X === 1 GELDS ) S)gre3?6^co y, 

n + 1 n v r 



1 



p.. a 






i 

Z 



r 
1 




, n = 0, 1, 2, . 

|SLcXfi (discrete branching process) ssodj'^. s5y& ^e [j3[c&c$> 




(n 



n > 

t> 

P.. 
jj 
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oo 



S53DC&&D = | 



2 a,, z' 



.0 



.*. [SSS i > So 



00 '"OO 

2 P.. z j = ^ 
j = o 



... 2.5.6 




X = 



n - 



X 







P z ) = 
S8 n > 1 So, X 



p CD z = S a 







P fz) = p. 

n \ / 3 



w 



X 



r 

X 

E z n 



X 



, 
z n+1 X ==j -P(X = 

n J j \ n 



j 



(2.5.6) 
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P 

n + 1 \ / _ n 




. P T (z) = P ~ 

n+l \ / n _ 



(2.3.2) ^|pg n-3D 



... 2.5.7 

_ n 






2.6 

9 O 






2io^^o gjfio 
^* ii) 

S)ScS 

8 




(i) j3#o gft 3 ^accessible state]: 



_ 

) 5 

u <S> q> co 



i &ot) j_sS5oS52j^> Si (accessible state] ^odjo&o. ^ "S>, |SS 



n & oSDOoSo^, P.. (n) >0 

13 

i _> j 






(communicating states] : 

x 

i &8 &oui ^STg' ^^ j ft, 5^ j ^S ?&oO i 

C5> Cp (P 

f S S^^) edged i, j fcdbo^b r5aaad r ^^bo: (comminicatng 

' 

' 

n, m 



states ^odr'. , o> S^PE?^ a&ocojS, p,, (n) > 0, p./ m ) >0 



ao 

i < > j 

* J 



n, m (< oo) o^ p..w = ^o. Pj , = ^o3, i, j 
x |^>ex> (non.communicating states) WO&P^. 







^a > 



(Symmetric property' : i < > j =$ j < > j 



(transition property) : \& i, j, k S 



,. 0, p. >0 

* j j i^ 






n, m > ^o4jcoi. s^^i (2.3.8) 



jS 

^cdj) s > 0) & &*_&$. ^o"g>, i - k. 




P k . tr) >0 65c&gd3to ,^ 36a># ^^SjpcpoS'o r & S'&^^sS^. eso"ei>, k 
, i > k, fc > i ^> i < > fc. 

CO 



(iii) cS^uPtf f&tf&e) (communicating classes) : 






cSaS"^. oSsSpo*^ 6KQ (communicating class) ^od^sSa. i 
fcdbo SsSxr'^Sh i 



(i) 



j S, i -> j -> j c (i) fe } c (i) => i -> j 




c = <>. 

ej 




1 fnbn-return') ftft wodj'^. c fi") oSij^ <d"^5 ^oa, i fed 

\ > <J> \ / O > 



j < > i addoto SiS* a,^ fcQ j & to*, i ^-^ i e5^dbo<a. <&) c (i) 

o co cj> eo \ ^ 

, i aa i 

* 



(return a 
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S 8* S^fib 3o<& oSsSpEptf &frf*6eo c (i) 3 c (j) c 
jf (mutually exclusive) tftftf&oS SC&*2>otfs$i3a 



k e dsfc^a && &o& &o~eb, i f N - k, k 



~ 



fc C7>^o- c (i) fif*a ]h |d c 
c (i), c (j) to o& &"! ^&*?tf 
c (i\ c (j) GO StfrS^ef ScSbjf 






i c (i), j i c (i) =* p () > 0, p..< m > = 0, 



So- p.. (n " = 0, p..( m/ > o<%i>to n, m, n', m' > 



S S 352o 




Q 






S = |l, 2, 3, 4, 5J rv Xo 
P ^ 







oo o 



P = 00010 



oo^o 



1 




f 



sSrS^ 



1 < > 2, 3 <-> 4, 4 -->5. ^o[_g-&3 ^o |_sSs^ 3 <- 5 
S a {1, 2, 3, 4, 



(iv) o^jjr dTdr/r^ (closed class) : Ss sy a.S' 
oA es tf^sg Borfa [_^S j |S L39o5S^S (not accessible) 
, es| cS&^tf tftf^SiO So^tf rftf/fd (closed class) 
a,g" &PCTtf tftf^S C ^oeoo$oa, [S i C, j g C 
n = 1, 2, 3, ... esox^ Ca 
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n ^ 



0, n = 1, 2, ... 



jjS3oaorfo"S>, tffiSotftfc 

(absorbing) tftfrfa 





(v) &d&3cb (irreducible) 

?5oe)o<So^)?6 





od3 CP 





S ^b, p.. w > 





n 

*'* 






S = |l, 2, 3, 4\ 










o ** 

p 

4 4 4 



y i 



\ 



S = Jo, 1, 2, ... , a- 1, aj. S & {o}, Jl f 2, ..., a- ij, J a J 



I, 3 
, a 

co cSoStf rftfrtdbeo. esjSD a.S' 




J& SsSfit), I^S i, j = 1, 2, 3, 4 oSb p.. (n) > aa>?dji a.* 

_ B J L J 37T JTj -'CO 



n > tf&*.sS^. s*8Dd3 S 



k a > 
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(vi\ e?<tf 0eT S^o; Aperiodic states^ : fod 

\ / JB V / 1> 







d > 8, d, 2d, 3d, 




Aperiodic state) ^ft>, d a i ^ScSD^^ wsS_tfc6 soSio (period) 



e$<>? esSj ^?6^cposex) n 

11 23 <X 3 




/"greatest common divisor") & (d. 8* ?Sp )?&) 

\ /\i _./ 



d, S n ^Aoi(^AfS<5 ft9cco^ 5 p t . cn ^ > 



d. 3 m sAoS$tfrfa<5 ^ox>^, p.. cm > = 0. 

i n 



d. s= 1 ycoD^ i feSS t9^^S_5^ (aperiodic) 



_ 





2.6.1 : i ft ts^ tf *5sosSao d. 



n ^: N &, p.. (n i^ >0 c&P6txi a-S" SSra'og'sS N > 

11 e> co o 



2.6.2 : p.. > sou^, n ^ N 



p .) .sj.,^, N > 6otooa. ^odSb^rfo'S), 2 3.8 



CO 



. _ p p n. ^ p ( p t.. 

J A J J^ Jk*^ J 1 H 

kS 



2,6.3 : i< > j ax>3, i, j S>^o ^5SS eJ6s'o'eo d. = d. 

" (S> i j 

fi >0 



11 11 



(N -f 1) s - Ns, (N + 2) s Ns, . . . o&> d, 






0, p., (n > > ^o&^do) n, m > 

CD 1J Jl C CD 



pjn+Ns+m) ^ (n) ( Ns) 
JJ j i n r ij 



p.. 



j cBtoS^ 65^^?6^eS5x> d. 
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o 



~n + (N + l) s + m" "n + Ns + m" = s 




o 



d, *3 s 






1k, d. 



d. 3 d. a 

(iu+Nr+n) g .^tfoga d. ^ d. ^ 5JAorfS$^a ^^i^^. Q& d. -' d. 

r * 



a,"! 

(j) 






/'class 



property') ^^)tfeo<S. S^Sd^Sb ^^.5 ^No^pepo^) ?6o8Do$oS)f6 




, n ^ N ^COD, P (n) ^a ^SSS &r>oS'&o p,. cn) > 
N 



9 4 ^ 



o o 



o o \ 



1 o 











o 



n =a 1, 2, 3, ... 

p(2n+l) = 







, n = 2, 4, 6, 



o 






2.7 Sc5<JS$tf &eu (recurrent states) 

so&o (finite steps) flf* a.S' ft 




* n 



(infinite steps) 
(infinitely often), iara^ ftdboSS) SS5^^cn 5ex>&*Co (finitely often) 








criterion &*To<3o&>. -^s [SSsSpesSSoo 







i 3 isSd Sjr^o^sSa n > 

L. 3 



.. = p f^<^ ^sa i ^^o^ ^oSScoa, nss 



P X i 



n 



P sSeSSS esaa i R>S?&oO fisdtfieoca. n 

a 4> 



=1, 2, ..., n- 
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'* 



N n 

2 f..tr> 



N _ , 



N 



S p 
n=0 

' . i 

N 



N 

N-N' 



2 f <* 
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(2.7.9' 



CO 



S = 
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n ( nr ) = 
P ik 



k,ET 

1 



fc -.ET 

r-1 



C n ) 
J 






kT 
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{p., j = j, 2 ..... 

distribution) 
(2.10.6) 
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' T| 7& SSj-ef*^iS ijSiSca^ a^)o03Si. T 



t , t T| 

. T 




f J ,1. 

(0, t" 



X SS'DOD, 63 

t 9 



t = 1, 2 ? 3, ... 
t > 







CD 

D (jS^cpcrex), l^iST 8 ^^Sjo) ?6o^S)oi5'^c 
t > d^^o s^sStSrj. (0, t" 
t > 
t 

|jSScSfi. 3!&Q3o<53 jjSJdSo.^ ^r^og' Seo^a (jS[8dfi (Integral valued 
process) Scs ^es>?6 [j$[J:& (counting process) && ^odpdb. t , t , t ,*..,t 





, 3 i5 .,..* n 



X=l, 



= 2, t < t <, t 



2 



x (t > s > o) 







X wcxag, JX \ 




^ 







Soj& (0, t 
23?5?6 ^^06 (Pure birth process) 







3.1.1 : (0, t" 
, t 



t 



(a) SCTESP 
, {X fc , t ^ OJ e^fl KraS L^u 



n- 

t 



a 1 i&po*, {x t , t ^ 0} ?& ^^?6 L s5L^ ^ aosaeso. (c) t 

x t ^ ?6^o^, {x t , t ;> 0} & 




(i) X 




(ii) 



t 



(iii) tj < t g wcooS, X t 

1 



(iv) 



X t . 

G 






3.1,2: 

, 55g JX \ 
(independant increments^ ^ 



(0, s", (s, t" oT* ^o^5S)o)?6 ?6o^oSo?3o ^o^^uo X , X X 

^ . ^8l 





X , X - X CX) 

5 &c?j*tf'es>o?So sS 




* 




fa) fif* 

\/ 






(stationary^ ^C3 ^s'Sdi (homogeneous) 



t t > o^b, X t - X t X t . t - X t . t co 

2 l' 2 I *" 






cS&o. (b) 





** "-J c) 



g SWJ*j-5 (jSj 
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3.2 






s, 






P(X = j X Q = i) V 




o 



| 



p(t) = 



. 



(i) p./^O, 



(ii) 



JES 



= 



Vt>0 
Vt>0 



.. 3.2.1 



(iv) p.. 
(v) 



(t) 



Lt 









J 



X u - 




. 



1 o (t) 



..= Lt 
13 

t _ 9. 



o 



n 





t 

l 




X.., X.. 



/transition rates^ 



. (3 2. 1) S* s = 



p. o+ A t) = 

11 



Pji ', 





kS 



p.*" p 

K1 



J- ...3.2.2 



(intensities' So- 



(3.2.1) (iii) S 



'3.2.1) 
v / 








...3.2.3 



82 d^&KSg JSj*er*k.li 



< l > D (At ' 4- t) tQ n (At) 

p ik p kj ^ p ij 



p.. ct) K) &&3S>, A t <* 

p..(t+At) _ p ..(t, p (At) p (At) 

p 



At ik At ij At 





vJ 

p./ (t) = S P, ft) V- P,. (t) X.. -..3.2.4 

. k ^J J JJ 
J 



(8.2*4) 

\ / 

SobS'tfe^ei} (Forward Kolmogrov equations) 
(3.2.1) ^ t= A s 




(As + s)-- y n (As) D (s) 

^ P ik P kj 
kS 



> A s 



(s >&ex> t 



V tt)BB 



fc^ j aex>SSSo (3.2.5) ?5&&P^ wsS^e^ o58bS'tfEse>& 

/Backward Kolmogrov equations) 




X.. 

1J 



X 00 X 01 



X io X n X ia 







* 
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, A 
(transition rate matrix) e5>o&r&. e &*[ fio^tfotf* (3.2.4), (3.2.5* 

' 



7 (t) = p(t) 

">- ...3.2.7 

P/(t) A P (t) 



P(0) " ( 8 i-) =I ^ ^^^^c^e (3.2.7) e& esfi^dSo^ex* (initial conditions) 

(3.2.7) 

\ / 



CO 

~n ! 




= exp~At~=l+ 2 _ , ...3.2-8 



n- 1 



sSj*L8g" esox^sSj^ (3.2.8) es^^ea ^^ es^j (3.2.4), (3.2.5) 

A Sg'^&'SS'n' $"& A ofl 



A = 




D, A &"[_&" <Sn^i- 35^ 23^(5" (eigen) Sexi^eo 
, D = 



/. P (t) =exp 



00 



Q (Dty Q~i 



n ! 

n=l 



co / Dt \n 

Q I + 2 -^rf- 



= Qe Dt Q-l ...3.2.9 
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n 



3.2.1 : 



Lt p (t) = TT , n S. S 



n 



n 



|jSLdBbe3b 
, n esffl 



CX3 



O r 
r 



nS 



t 




CO 



n 



0, 



(3.2.10) escr'J)o!T (3.2.4) 



0=2 



J JJ 



[53* > 




n S 8. 



- 9 t ^ 



...3.2.10 



jS ...3.2.11 



.*.8.2.12 




X. 

CO ij 



(3.2.12) So 




(0, 0, 0, ...) 
o (3.2,12) 




|jS[cx3c> ^Positive recurrent Markov process) 
TTT^; n == o, 1 3 2, ...J Jy^sf 





3.2.2: 




8S 



p (0) = TT ; n = 0, 1, 2, ... wa3 f p ct) = 

n n B 

V t, SeffidSo^a Sp5 8 i_S *\oo*ofir* adWboflrS tfo^off 8 gj*e\ 

U C/ (2* tJ 



3.3 oj^u^jsr'^ L*>L?O*U [foi^son process 

*" - (^ ft t *. 

( 9 _ 00 rOOcpD O rODSpg { cOP _cO, | t , | 

s < t S, 



"X = j X = i" 

_ t s _ *-* o 




p (s, t) = P (X t = j X s = i) (i, j = 0,1,2, ...) ...3.3,1 



S.S.I 





(a) |X t , t ^ OJ 



(b) L.SS8 ^g^ ^^^ A 1 * 

, X 





CO 



(ii) 2 





og P.. (t, t + ^ t) = i x A t + o (A t) 

A * > 





At 



o wDib&j o t a 






(i) ^|"J9 (t, t + A t" 

i ^ i + 1 65^ o[^&o ?5o4JS)o^ oSo^sS|^ X ^t + O M t) 




O 
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postulates'! 

r J 

P Poisson events) ofc> eodj*&. )S^ (s, t) 
?5ogp<? X X eso^tfo ST^, t s 



o {X 

CO (^ t 



, t 



3.3.1 
t S: OJ & X (> 0) s 



t 



, t ^: So, 



\ / n = 1 

*-Vl 



X , t ^ 0| 



, n.O, 1, ... 



$o eb, X X , X X a) a*"! 

|j5[di) 



3.3.2: ^COD^S" fe^^e^o^ S*d&, p Q co) = P(X 0) 
Sb JX f t ^ OJ 





(a) S) 
(3.2.1) e^tforr, jjSd s < t < u 






*0 = S P..fs, t)p. ft, u) 

/ ij \ / jn V. / 

j 



, t, u eb 83d5cjeorv 2tf?Srp 0, t, t + 



PX - 0} s= 

.07 a 

P 0n (0,t + A t) = S P 0j (0, t) p (t,t+ At) ...3.3.3 



J 

n So, 

P 00 (0, t + At) = P 00 (0, t) . POO (t, t + At) . . .3.3.4 

n > So (3.3.3) 



At ) 

n 1, n 



n (0, t) 
(b) ^^g^ (3.8.4), (3.3.5) 



p(t + At) = pt) -i _ X A t + O At ..3.3.6 



o 






0(At)' +0(At) (n>0) ...3.3.7 



(3.3.6) 

p (t+At) _ p (t) OfAtf 

K o F o i ( \t-* i 



r = 



At u At 

d 



dt 



(p Q (t) ) = p Q ' (t) 




> 

At 



p o '(t) = _ x p') ...3.3.8 



(8.3.7) 



p '(t) = _^ p ( t ) + x p ( t) f n >0) ...3.3.9 

n n n-1 v / 



88 




<*> 



= 



po. (3.3.8) 



(3.3.8), (3.3.9) 




(8.3.9) 



p = e -xt 



...3.3.1C 



p '(') = y /(t > e~ xt 



n n 



- X V e 



- xt 



n 



= V't 1 ' - X V 



n 



3 e t 



n 



(t) - V 



V f c^ = X V 



n-1 




t = 



t 



V>+C=\ 





C = v <o> 

n 



*/ 





n 1 



=l 2 

i, , 



X ^ V dt - Xt 




V Q > - 1), 



t 

o 



o 
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t 



v (t) - x V u dt = 



n n - n ! 







n (t) = V n e- xt = e" xt n=l,2, 3, 3.3.11 

n n n ! 



(3.3.10), (3.3.11) e> |jS 



t ^ 



t = n) = e- u , n = 0, 1,2, ... 



(Q 9 t" 

9 \ 

= X t- 



t = 

(s > 0) SS^ L^d'o^&cxoS, X g = i 




p, 

C 11 " 1 ) 



= , n < i, J 



(truncated poisson distribution) 



...3.3.12 



P , p. 
e n in 






Pi, (*!' ( 2 ) = P ( X t 2 - J X t, - 



*. 



P ( X t = J' X t, - i) - 

6 1 




X o = 



P (X t - X, = j - i) 



~ xt 



1J 



= o 



J 



...3.8.18 



(3.3.13) &o>, LSSa j > i 8, 



Lt p. () = 



t 



tf SS3 






sdJ 



(3.2.2), (3. 3.] 3) o 



X.. = Lt 

n 

t * 



1 



t 










_ e _xt 
Lt _ = X 



t 



X.. = Lt 



= Lt 






Xj. - X, 
*= 0, 



j = i + 1 



j - i > 1 . ' 



(3.3.14) 



(3.2.2) |Jj 



X,., X,. 






t 



3 8 .2 &o'tf c jo$' 



distributions) : 



t 



<3&*att$<&j /inter-occurrence times 
= 0, t^ t ... 



, = t 1 - t , T = t t 

O I I 1 Q j j 



2, 3, ,.. 

(a) L55*tfo t. So 
^S, T-.T^* ... co 

ola 

sSgsS^eoHnter occurrence times) eodj*6:>. &88 




3.3,3-: 
T , T g , ... 8ax)ea&^ (negative exponential) 




t ^ 8, n = 1, 2, 8, 



T !> t ^ofio^ X i X = 



/. P " T > t " -= P " X t . - X = 

- n - 



P " X = 



X , ssog'&tf&>&o Xt 




" " xt 



P 'T > t" = e 

_ n 



t ^ 0, n = 1, 2, 3, ... 



P ^ T <- t ) = 1 -e" xt -..3.3.15 



T o ?oo^s$g^ ^O^SP jjS&cXfisSDD (probability density function) 



{ P ( T ^ t ) 1 = X e- xt , X > 0, t ^ 0. 
dt I v n ; 3 



{X, t^ 0} 




t 

o 

R S*&, t 

" 



62 





o& 




R , S o 






n 



o 



i 



4,*- s v 



n- 1 sS 



R 



n 



n 



R , S O 






3.3.4 : R 



C 



P (R t ^ x) = 1 - 



x ^ 



...3.3.16 




R ^ x 



- t 



_ n 



t ,, t o 

Ii-ln 

(t -t , t" 
t t 



t 



+x T >t t 

n 



n 






P(R 






P " T >t-t 

n n~ 1 



3.3.3 



P ( T ,^t) - l ~ e - Xt 



. P (R ^ x) = 1 - 



- e 



~ X ' - 



X(t-t . 

\ n I 



93 
1 e~ Xx , x S: . 



3.3.5 : S 

t 



P (S t <: x) = 1 - e- xx , < x < t 

^* i J. JtM. I 

P (S t = x) = e- xx , x = t \ 



(0, t 
(3.3.3) 




(0, t" 8* S'fctfo fib^j. c6o^>&)SS SSQDD^ ^o^So^oi^) ^^g^otS'SSxi . x 
(t x, t" 8* ^fe?6o 
|jS x ^ 




(S t <, x" = P j (t - x, t" 

1 



= 1 P (t X, t" 

1 



= 1 P 



3-S.3 




a. T, 

|jSS s, t ^ e-)& P(T ^ t + s T > t) = P(T ^ s). 
a^^-, <c sSoS>" (forget ful) 
(0, t" ar 8 ^d?6 6oatf^tf 3DS)^ s t 



f s 




o a 



J &*&<$<&> (waiting time distribuiton) : 




8 

. n 
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n 
W a*&-&1>,W =2 T. fecrorfo (3.3.3) [SSs'tfo T, 

n -. n r Q \ / u r 

r = 1 



= 1, 2, ..., n) co a,li s^o 



n 







5e>a SP^ Sgres^Sg) ?6o^o^ o^eao (additive property) 




W c3bj^i_ S)^a3^^o n, X Soao&e>S r?^ (gamma 

n Cr \ J 



f w (t) =: - t*-i e- xt , X > 0, t > , 

n n ! 



U 33.18 



I 

n n .< 

E(W } = , Var.fW } = 

\ TO./ \ \ n/ 






. t c&coo :g n 
t S3^dSr^ cSSaS cb$o r6o S"fcr6o n 



t" = "X t S: n". 



n 



( xty 



>-Xt 



J ! 




t tft 65^53 o?6o ' W 



00 





i * * 

J = n J - j = n 
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3.3 A J 



(uniform) 



3- 



3.6: (0, 

V 



dt 
P(t<T<t + dt X T = 1)= - , 0<t<r .,,3.3.19 

\ J y 



P 't <T< t + dt, X T = 
P(t<T<t 



= P (t < T < t + dt) . P (X T = 1 T = t) ~ P (X r = 1) 



^ - Xt , -X(r-t) /r , XT 
Xe dt . e v / + \ T ' e 



dt 



3.3.6 

o 



T" fi^ t, < t. < ... < t < T 69<5)jfi3di> t , t 9 , ...i t o 

_12 n Ocol^i n 

o&<b?5ex> jSo8faon-dSo^ ^JiaS'o^, fO, r^ 8* t , t , ...,t OD 

\ 1 a H 



O 

n d^S) s*oS)o2i^)eo ^)^ODD. ^CT'O^O 3.3.6 ^gSc'gj t^ t g , .. t^ co 




(0, T" fi^ SS'fij'SS S)5Se3R?^ w&fSeoS n ^& ^c^g^oD (Order Statistics) 

(0, r" fi^ ><* aa- &5&SaoS n 

, ..., U 








U , .. M U o& fgogDo^o n 




,.* 
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n ! 

n ^ *r ^ 

t 



f t t fx , ..o x ) = _. , <; x- 

1' *** n n T 



x ^ T . C3.3.20 

n \ 



n 

, fr,, T + A T" ....... 3 f T , T + A T $ & n 

\ 1 ^ A 1. \ n n *-* n. 



, 
... XA 

^"-^ Tl 



-XAr -Xrr-^r. ... AT 
r e n e ^ , l " 



n ! 

T i A 



t. < t, <...<" t o J5^ ^f5n n 

la no 



X T 



n ! 






t f (T-3 ... T ) A 1"., ... A T 

LJ *""' n n n 



n ! 

o T 



, I 



T ^ r . ...3.3.21 

n 



(3.3.20), '3.3.21) o tf>|pj '0, r" . ^as^ eSo n 

?Soooao)^ 






t l9 ... t sStf.n 
T n ' ! n 

X$Oi)o!)i\F D c3ir o , ^3 t 15 * t O 

-i i n 





(hypothesis) 




(0, r" * n 

, ^s n ?5o^6)^ai ^CP^ Q <t { < ... < t^ < 
t , ... 5 t 

1 n 



(0, r" ficrSj aspsss?^ !ao n S 

t, t ..... t e-0 t r" flf* n 




oJ 



(0, T" ^ t' eo n r 

2 

T 3 V (t\ T 



12 
D 



S = 

n . 

1 = 1 



n r __ , M . - - _ ^ g 



, E (SJ = _- , V (SJ = -- , s^ n 



oJ 



t 



. t S SoJb &" t = 0, 1, 2, ... 






^ P (0 <: p ^ 1) ^^, atfrfS JSo5J.Sgfi& q (= 1 - p) 

fio^^^a. S^oLO-oO a^^odJ'^o. (0, n[ 
X von j, S8tf oJS X 




' n 



P (X = x) = 

\ n / 



x = 0, 1, 2, ... n 



IX \ 

' n J 




t = 0, 1, 2, ... o 

' ' 



Q 






n 







n S5 



n 



W = 




(n- 



n 3 ?5o6)dD^ X sS 
n 



n 



' X ~ 1 ' 



.- & 



P (T n = x) = q^' 1 p , x = 1, 2, .. 



SS 



* 

w 



(x -1) 

"Saotffi (x 3) 







n -l 



n 1 



. p 



' x 1 

, n- 1 



p n q x-n 



' -n * 



P n - 



6i & n, p 






SSfi ^Negative Binomial) 



J 



X , t 



0'', 



1 p= q 



'II 




I tf&'o ?5c&do?SrT>&>, ^ (jSS^SjL f$oB 8 5$|tfS* II 




(0, t" 
X <l> f X ( ^ c^ sSj-a*. X =X (1) + X 

EL * fc 



3.3.7-.- 'X^', t ^ oj, JX^), t ^ 0} 

Xp, Xq o'be&oi5f 8 S Ti os' 1 S' [cS L 8d8oeo. 




P (X t < = n, X t ( ' = m) 



00 



= n, X t <2) = m X t = j), P(X=j) 



j ^ n + m 

P 



i> t 



'. P (X t < = n, X t < 2 ' = m) . - 

- p (x t ( i ) = n, X t t2 > = m X t = n + m) 



X P (X t = n + m) 



(0, t" T* a + m fSo&forSeo ^o4f3oacJ3;)^) T> oi3&3. SaiJfi 'SoJfo 

p, 1 P ?So55s5^o5* foo4JD, I d's'aS ^o^oSoa n, 
m 

' i t 

n + m 



n 

k. * 



P (X ' = n, X t 2 ' = m) 



n + m (Xt) n+m 

P B (l-P) ffl .- xt (n+m) ! 




n ! m 



100 



oo 



P(X t "> = n) = S P (X t < = a, X t < a > = m) 




n 



2 



(Xpt) n 



n! 



m 



m! 



, t 



Oj co BO&P sStff6rp Xp, X(l p) 






3.3*6 ^cu?^ ^ ^5f3c&S5 &&*&$&&&> ^generalization) : 




fnon-homogeneous N 

\ y 






non-homogeneous or non-stationary) 5>ox>a*' isS c& 



3 3.8 




(0 



, t 



( * *\ /* j. i 

m) (t.t + 





X (t) At + O (At) 




(iv) (t, t + A 1 " * a.&SS'o'eS) Z&fr.$ fSo&dotfoi adtf 
0(At). e 






Lt 



At 



Xt ^ fooo^sSoo X, m(t) = X ^s) ds 

c 




P fX - X = m^ = e~ / m < t+s > - 
v t+s t y X 



n 



' 




. (3.3.22) isS^cfo X t+a - X t ^ (3.3.23) 
m (t + s) m (t) n 8 rf'o ^o^S* S^ea*^^ 
m (t\ ^ |jS[Sd& (^S^. rtto si^og ijSS>ci&fia (mean value 
function) esofij*&. Spoxi^S' ^sS[_8d8oe* X (t) = X, m (t) = Xt 

3.3.2 



o 



p (0 = P / x _ x = 

r n v. t+s s 



p (t+At, _p/x A -X =0) 

V s+t+At s / 
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&*&cSg 





P {(s, s + f <5* 




, (s + t, s + t + At' 



= P /(s, s + t" * tf 

X P {(s + t, s + t + At" 6* 







1 X (s +t)At +O(At)" (V 



p 



(t+At) _ (t) 



At 



At > 



log 




At 







t 



X(s + u)du 







X (y) dy 



^B 





i 
fe 



p = e -f m(s+t) 



n > So (3.3,23) 







(B) Socfo j &*oMfipg" |S|J<ai) (Compound Poisson Process) : 



3.3.9: X 



OJ. 
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, tS:0 So, 



N t 

x = 2 y, 



...3.3.24 




i. (N t , t ^ 0} 
ii. Y . Y . Y , 



oo e/3 




(a) Y. = 



Y : i s$ 



X : t 
t 



(X , t ^: 0} 



CO 



Y. : i 



X : t ?O^ 



n 



^033^ 






= E{E ( X N)} 



1 



} S 



t ^ O 



J 





i 
t , t ^ 0} 




N 



E(X N t = n) = 



Y. 



i= 1 



n 



= E { 2 Y.} (V Y 1 o>, N t 
1 



eo 



104 srjc$g 



E ( X t N t) ==N t E ( Y 1 ) 



( X t) - E ( N J ' E 



= Xt.E(Y 1 ) ...3.32S 



3 S;j8 (conditional variance) & 

Var. (X t ) = E { Var. (X t NU + Var. |E (X t N t )} 

D 

s>3, Var. (X N t = n) = Var. / Y. J = n Var. (Y 



Var. 



(X t N t ) = N t Var. (Y 



.'. Var. (X t ) = E (N f ) . Var (YJ + Var. "N ( E (Yj" 
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rf ?5&dSp 



l05 




jg;6;S ij$ Sd& (pure birth process) rr SS&). 9-^ 0, t" 5* 
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